System and method for optimizing a graphics intensive software program for the user&#39;s graphics hardware

ABSTRACT

A system and method for optimizing the performance of a graphics intensive software program for graphics acceleration hardware. This system and method encompasses a procedure that validates the different functions of a 3D acceleration capable video card, decides whether to use the acceleration hardware and optimizes the software application to selectively use the functions that work on the specific video acceleration card. Functions checked include sub-pixel positioning, opacity, color replacement and fog. If these tests are successful, then the graphics acceleration is used by the software application. However, if the tests are not successful the decision is made not to use graphics accelerator. Those with ordinary skill in the art will realize that it is not necessary to perform all of the tests in a specific order. Additionally, other types of tests could be performed to ensure software application and video card compatibility before the software application is uses graphics acceleration to render 3D graphics.

BACKGROUND OF THE INVENTION

1. Technical Field

The invention is related to a system and method for optimizing agraphics intensive software program for a user's computer graphicshardware. Examples of such graphics-intensive programs include flightsimulations, computer aided design programs, photo-editing programs,computer games, and clear type and presentation software.

2. Background Art

Due to advances in graphical operating systems and 3D computing, therehas been an enormous increase both in how much data is sent to computermonitors and the sophisticated calculations that must be done todetermine what is seen on the screen. As a result, computer video cardshave evolved to be much more like co-processors. The video card in apersonal computer plays a significant role in the following importantaspects of the personal computer system:

Performance: The video card is one of the components that has an impacton system performance. For example, application software that depends ona high frame rate (how many times per second the screen is updated withnew information) for smooth animation, is often impacted far more by thechoice of video card than by the choice of system Central ProcessingUnit (CPU).

Software Support: Certain programs, such as, in particular, video gamesand other graphics programs, require support from the video card. Someprograms, such as 3D-enhanced games, will not run at all on a video cardthat does not support them.

Reliability and Stability: Choosing the wrong video card can causeproblematic system behavior, such as computer system crashes and hangs.In particular, some cards or types of cards are notorious for havingunstable drivers, which can cause a host of difficulties.

Because the computer screen is two-dimensional (2D), everything that apersonal computer displays must be two-dimensional as well. In order forthe computer monitor to display three-dimensional (3D) objects, it isnecessary for them to be converted to 2D images. This requires specialprocessing and a large amount of computing power. The push for morerealism, more detailed graphics, and faster speeds in such programs asaction games, flight simulators, graphics programs, computer aideddesign (CAD) applications and presentation software, means that more 3Dwork must be done in a shorter period of time, requiring the use ofspecialized 3D accelerators. To perform the large amount of computationwork necessary to translate 3D images to 2D in a realistic manner,accelerators were designed that did much of this work with specializedhardware, instead of forcing the system processor to do it. Using a 3Daccelerator allows programs to display virtual 3D worlds or 3D objectswith a level of detail and color that is impossible with a standard 2Dvideo card. Common 3D operations wherein 3D accelerators are usedinclude:

Gourad Shading: This is an algorithm that is used to give 3D surfacesrealistic shading. The effect helps the object appear to have depth andhelps to define the shape better.

Clipping: This operation determines what part of an object is visible onthe screen and “clips out” any part that the user cannot see. Parts ofobjects that are off-screen are ignored, thereby improving systemperformance because the off-screen portions do not have to be computedfor rendering.

Lighting: Objects in the real world have their appearance shaped by thelight sources in the scene. Lighting effects cause color shading, lightreflection, shadows and other effects to be added to objects based ontheir position and the position of light sources in the room.

Transparency/Opacity: Objects in the real world are transparent,semi-transparent or opaque. These transparency/opacity features areemulated in software/hardware.

Texture Mapping: For realistic objects, it is necessary to overlaypictures on them to give them texture. Texture mapping allows objects tobe made so that they appear to have substance instead of being “flat”.There are several different types of texture mapping that are used byvarious software and hardware.

Dithering: This is an effect that is actually used in many differentplaces, including regular 2D graphics and also in printing. Dithering isthe process of mixing a small number of colors together in specificpatterns to create the illusion of there being a larger number ofcolors. In 3D, it is used largely to show more realistic color withoutneeding to increase the color depth of the image (which means morecomputation time and more memory to store the graphics). Dithering takesadvantage of the eye's tendency to blur spots of different colors byaveraging their effects and merging them into a single perceived shadeor color.

Fogging: Fogging serves two purposes by blurring objects that are in thedistance. First, it helps to make the scene appear more realistic.Second, fogging allows the 3D process to be performed more quicklybecause those objects in the distance that are “fogged out” can becomputed more quickly since they are shown in less detail.

Filtering: There are several types of filtering that can be applied toan image. These are used to “clean up” the image and smooth out texturesand shapes. In particular, bilinear filtering is used when showingtextures up close to remove the “blocky” look that results frommagnifying an object when showing it at the front of a scene. Bilinearfiltering refers to a process where the graphics hardware smooths theappearance of a texture by interpolating from one pixel value to thenext. Textures appear perfectly smooth, even if viewed from very close,thanks to bilinear filtering.

In order to benefit from 3D acceleration features, it is necessary forsoftware to know about them and support them. Unfortunately, with 3Dbeing a relatively new field, a whole new breed of different andincompatible video cards has hit the market. Different cards havedifferent capabilities. Additionally, different cards will behavedifferently—that is, the same instructions given to various video cardscan yield different results. Support for a particular card is requiredfrom a program if it is to take advantage of the 3D card's features.Otherwise, the program will not benefit much (or at all) from the 3Dhardware. Most of the specific support for 3D hardware is from games.This comes usually in the form of special versions that have beentailored to different video cards. In addition to a regular version of apopular game, a version may be created by a company to support aparticular 3D chipset. This version will usually either have muchsuperior graphics, faster performance, or both. However, it typicallywill not work with a different type of 3D card. Fortunately, newstandard libraries are being developed to tackle this problem. Driverlibraries like Microsoft Corporation's Direct3D® and OpenGL are designedto allow software to be written generically without tailoring them toeach 3D chipset on the market, allowing them to be used regardless ofwhat chipset is used. Software applications that are designed to usethese libraries can avoid some of the need for customization.

Many applications employ 3D graphics and 3D accelerators, as well asdriver libraries such as Direct3D®, to improve the performance andreliability of the application software. One such application, forexample, is presentation software, such as Microsoft Corporation'sPowerPoint®. PowerPoint® has a capability of providing a “slideshow”feature wherein a presenter can present a presentation in slide form toan audience using a computer.

Customers desire great performance in slideshow when they present. Theywant the animations to look as smooth as the ones they see ontelevision. In order to do this, the PowerPoint® presentation programuses hardware graphics acceleration provided by the Direct3D® driverlibrary. Unfortunately, because some video cards support certaingraphics features, while others do not, when the application softwaretries to use them, they do not work properly. For example, some cardscrash or hang on certain function calls. Obviously, it is extremelyimportant for the slideshow feature to work reliably. If a game crashes,the user can reboot and start again without much issue. However, if aslideshow crashes it is another matter. For example, if a salesperson isgiving a presentation to a large group of people and the slideshowcrashes, the audience may consider the presenter incompetent.

SUMMARY

The present invention overcomes the aforementioned limitations in priorcomputer software and hardware systems by a system and method that seeksto get the best software application performance and reliability byoptimizing the application software, such as PowerPoint®, for thecapabilities of specific video hardware. This system and methodencompasses a procedure that validates the different functions of a 3Dacceleration capable video card, decides whether to use the accelerationhardware and optimizes the software application to selectively use thefunctions that work on the specific video acceleration card. Thisprocedure is typically run every time just as the application softwareinitializes. The invention can be couched in the terms of use with agraphics presentation program, such as PowerPoint®, and a graphicsaccelerator driver database such as Direct3D®. However, those withordinary skill in the art will realize that the system and methodaccording to the present invention could be used with any graphicsintensive application program that employs the use of graphicsacceleration hardware in combination with any graphics driver library.

By way of example, a software application, such as PowerPoint®,typically interfaces with a 3D graphics accelerator via a driver librarysuch as Direct3D® to more efficiently process graphics displays. In thecase of the PowerPoint® software application, a slideshow module isincluded which presents the “slides” in the user's presentation as acohesive package. This slideshow module interfaces with a drawing layerthat provides drawing commands to display these slides on a computermonitor or other display device. The drawing layer can interface withthe Central Processing Unit (CPU) of the user's computer either via asoftware renderer to provide the drawing commands, or can interface witha graphics accelerator renderer, such as the Direct3D® renderer, whichinterfaces with the graphics accelerator to draw the respectivedrawings.

The graphics accelerator renderer, which typically resides in theapplication software, interfaces with the graphics accelerator hardwareand software through a drawer, which converts drawing layer commands tographics accelerator commands (in this case Direct3D® commands).Additionally, the graphics accelerator renderer initializes the graphicsaccelerator hardware and software via an initialization module forcompatibility testing of the graphics acceleration functions anddetermining which functions are to be utilized with the applicationsoftware. The present invention automatically performs compatibilitytesting with the graphics acceleration hardware and software as part ofthe initialization process. It should be noted that it is typicallydesirable to use the graphics accelerator hardware rather than thesoftware with this invention, since this hardware processes 3D data muchmore quickly than either the software resident in the graphicsaccelerator or the software renderer that interfaces with the system'sCPU. The test module also interfaces with a drawer and texture managerfor the purpose of performing the aforementioned compatibility testingof the graphics acceleration hardware of the video card.

In the most general sense, the invention performs compatibility testingof the graphics hardware by first checking to see if graphicsacceleration is selected or enabled, checking if the video card is on alist of cards with known problems, and verifying that a graphicsacceleration driver library initializes the graphics acceleratorsuccessfully. The invention also checks to see if sufficient videomemory is available to perform the various compatibility tests andverifies that calls to the video card hardware are successful. In eachof these process actions if the particular process action indicates thatthere is a problem with the compatibility of the video card and theapplication software the decision is made not to use the graphicsacceleration hardware of the video card. Additionally, if the aboveprocess actions indicate no compatibility problems exist, the options tobe used with the video card are selected and stored. Then sub-pixelpositioning is tested. If this test is successful, then pixel offsetvalues are stored. If the test is not successful, the decision is madenot to use graphics acceleration hardware of the video card. A test isthen conducted to determine whether the opacity function performscorrectly when used with the graphics accelerator. Again, if the opacityfunction does not work correctly then the procedure is exited with adetermination not to use graphics acceleration. If the opacity test ispassed, however, a test is conducted for color replacement. If this testdoes not pass, another test wherein a fog function is used to performcolor replacement is conducted, and if this test also does not pass thenthe decision is again made not to use graphics acceleration. If thecolor replacement test passes, then the decision is made to use graphicsacceleration, and the system and method according to the presentinvention continues to run the application software. Those with ordinaryskill in the art will realize that it is not necessary to perform all ofthe above tests in a specific order. Additionally, other types of testscould be performed to ensure software application and video cardhardware compatibility before the software application is used to render3D graphics.

By way of example, the software program, in this case PowerPoint®,typically initially examines whether the user has turned off thehardware graphics acceleration option. To this end, a graphical userinterface may be made available to the user to disable the graphicsacceleration option. If the user chooses to disable the graphicsacceleration option, a flag is set in the software. The applicationsoftware may also set this flag if a previous attempt to graphicsacceleration fails. The application software may then examine whetherthis flag is set in the software. If so, the system and method accordingto the present invention determines that the graphics accelerationhardware of the video card is not to be used, and exits the procedure.If the user or software has not turned off the hardware graphicsacceleration option of the software, the system continues to examine thefeasibility of using the graphics acceleration hardware of the videocard.

In one embodiment, the software application then examines whether thevideo card resident in the user's computer hardware is on a list ofcards that is known to be incompatible with the software program. Thismay be performed by the application software by maintaining a databaseof video cards and/or video drivers known to be incompatible with thesoftware and checking to see if the video card and/or video driverresident in the user's computer is in the database. If the card and/ordriver is determined to be incompatible with the application software bythis action, the application software does not use the graphicsacceleration hardware, and exits the procedure.

The application software then attempts to initialize the graphicsacceleration hardware by setting up the drawer in the applicationsoftware, ensuring it loads successfully and making calls to it. If theinitialization is unsuccessful, the system determines that the graphicsacceleration hardware is not to be used and exits the procedure. If theinitialization is successful, the video memory of on-screen andoff-screen buffers of the video card is then checked by requesting thevideo memory for them. If this call is unsuccessful the decision is madenot to use the graphics acceleration hardware. If the call is successfulthen the system continues its checks.

The system then requests to access the video card hardware through thehardware abstraction layer (HAL). The HAL is a software interface tovarious 3D features of the video card. If the hardware device cannot beaccessed, the decision is made not to use the graphics accelerationhardware. If access to the graphics acceleration hardware is successful,the system and method according to the present invention proceeds to setthe graphics acceleration options necessary to support the applicationsoftware. These options are saved in memory and accessed as theapplication software runs. Such options include, for example, howtextures are to be drawn and the selection of various filtering options.

The system then checks if there is sufficient video memory to performthe remainder of the compatibility tests between the applicationsoftware and the graphics hardware. During these tests, test patternsand computations are written to off-screen video memory. Hence, both theon-screen and off-screen buffers are used. To draw the test patterns andto perform the applicable calculations of the present invention, theretypically must be enough video memory for the on-screen and off-screenbuffers plus the number of textures used in the compatibility tests (ina tested embodiment this was 8 textures of size 128×128 pixels).

The system then creates a texture, filling it with a test pattern. Thetest pattern preferably consists of a black background and diagonal lineof pixels of all white color except the first three pixels which arered, green and blue. This test pattern is typically drawn to theoff-screen buffer at position 1,1.

Next the sub-pixel positioning is tested. Sub-pixel positioning isimportant to presentation software, such as PowerPoint®, because theclarity of text on the computer screen is critical to the readability ofthe presentation. In general, this sub-pixel positioning test consistsof 1) drawing the test image to off-screen video memory; 2) determiningwhere the pixels for the test image were drawn using the expected pixelboundary and color; 3) calculating the offset values between where thepixels were expected to be drawn and where they were actually drawn; and4) determining whether the offset values are acceptable to correct forthis offset during 3D graphics rendering. If the offset values are notacceptable, different offset values may be tried until the offset valuesare below some predetermined threshold. This process may be repeated formore than one cycle.

Specifically, for the test conducted by the system and method accordingto the present invention, the test pattern is preferably drawn tooff-screen video memory and it is assumed that the center of a testpixel is at 0.5, 0.5 (further assuming a pixel matrix of 128×128 pixelsand having an origin of 0,0). The pixel values of the test pixel arethen read back from the off-screen video memory. If the values matchwhat is expected (i.e, the values are below a given threshold) then thissub-pixel positioning is used (stored in memory to correct for pixelposition during the application run time). That is, if the difference incolors from what is expected and what is read from the screen buffer isless than the threshold, then this sub-pixel positioning is used. Oneexemplary formula that may be used for this purpose is(abs(r_actual−r_expected)+abs(g_actual−g_expected)+abs(b_actual−b_expected)),where r, g and b represent the pixel colors of red, green and blue,respectively. For example, using a threshold of 24, if a RGB color of(255,0,0) is expected and an actual color of (250,1,0) is read, then thedifference in colors between the actual and expected values is 6 so thissub-pixel positioning is accepted. If the read values do not match(i.e., the values are over the threshold), then other pixel offsetvalues are tried in an attempt to determine the actual pixel offsetuntil a match is found. This process may be repeated for a number ofcycles or iterations. In a tested embodiment four cycles were completed,using expected pixel offset values of 0.5+{fraction (1/16)}, 0,0+{fraction (1/16)} and 0.5 from the origin of each pixel center asdrawn to offscreen video memory by the graphics hardware.

If the difference in value between the expected pixel color and the readpixel colors is greater than the threshold (i.e., 24) for all desiredcycles using different offset values, then the minimization filter andmaximization filters are turned off and the desired number (e.g., four)of cycles of the above process is repeated again. The maximizationfilter is used when an image is being increased in size. It usesinterpolation between pixels so the image looks smooth (usually bilinearfiltering). The minimization filter is used when the image is decreasedin size. It also uses interpolation but typically uses a differentalgorithm than the maximization filter. If turning off these filtersstill does not bring the difference between the expected pixel color andthe read pixel values under the threshold then the determination is madenot to use graphics acceleration as the resultant sub-pixel positioningwill result in noticeable distortion or blurring of the 3D image.

If the sub-pixel positioning is determined to be suitable, the systemand method according to the present invention then tests to see if thegraphics acceleration hardware can perform the opacity function. This isdone by setting the opacity of a pixel to a prescribed value such as50%. The color value of the pixel is then read from the screen buffer.If the color has not changed then it is assumed that the card cannotperform the opacity function and graphics acceleration is not used.

The color replacement function is then tested. Several conventional 3Dacceleration techniques can be used for this purpose. One of thesetechniques tests color replacement by replacing all of the color in atexture by another color without changing the opacity values. Forexample, a texture is drawn in red by telling the video card to useDiffuse Color for the texture and to ignore the original color of thetexture. The color of the pixels is then read. If the color is not asexpected then an attempt is made to determine whether a fog function canbe used to mimic the opacity function. If this fails, then graphicsacceleration function is not used. However, if all of the above stepswere successful then the hardware graphics acceleration function isused.

Even with the above-specified compatibility tests, there is a chancethat the hardware will fail. To mitigate this possibility, the presentinvention automatically disables graphics acceleration if the graphicsaccelerator hardware crashes or hangs during the running of the softwareapplication. The next time the user launches application software,slideshow in the case of PowerPoint® application software, it does notcrash. To do this, a flag is set when slideshow is started and unset itonce the slideshow successfully ends. However, if slideshow does notsuccessfully end then the flag is not unset. The next time slideshow islaunched, the program notes that the flag is set and graphicsacceleration is not used.

DESCRIPTION OF THE DRAWINGS

The specific features, aspects, and advantages of the present inventionwill become better understood with regard to the following description,appended claims and accompanying drawings where:

FIG. 1 is a diagram depicting a general purpose computing deviceconstituting an exemplary system for implementing the present invention.

FIG. 2 is a schematic of how an application program interfaces with a 3Daccelerator.

FIG. 3 is a schematic of how the 3D software renderer interfaces withthe graphics accelerator hardware and software.

FIG. 4A is a flow chart showing the general process actions of thepresent invention.

FIG. 4B is a continuation of the flowchart depicted in FIG. 4A.

FIG. 5A is a flow chart showing the process actions of the presentinvention in more detail.

FIG. 5B is continuation of the flowchart shown in FIG. 5A.

FIG. 5C is a continuation of the flowchart shown in FIG. 5B.

FIG. 6 is an exemplary graphics user interface for the system and methodaccording to the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

In the following description of the preferred embodiments of the presentinvention, reference is made to the accompanying drawings, which form apart hereof, and which is shown by way of illustration of specificembodiments in which the invention may be practiced. It is understoodthat other embodiments may be utilized and structural changes may bemade without departing from the scope of the present invention.

Exemplary Operating Environment

FIG. 1 illustrates an example of a suitable computing system environment100 on which the invention may be implemented. The computing systemenvironment 100 is only one example of a suitable computing environmentand is not intended to suggest any limitation as to the scope of use orfunctionality of the invention. Neither should the computing environment100 be interpreted as having any dependency or requirement relating toany one or combination of components illustrated in the exemplaryoperating environment 100.

The invention is operational with numerous other general purpose orspecial purpose computing system environments or configurations.Examples of well known computing systems, environments, and/orconfigurations that may be suitable for use with the invention include,but are not limited to, personal computers, server computers, hand-heldor laptop devices, multiprocessor systems, microprocessor-based systems,set top boxes, programmable consumer electronics, network PCs,minicomputers, mainframe computers, distributed computing environmentsthat include any of the above systems or devices, and the like.

The invention may be described in the general context ofcomputer-executable instructions, such as program modules, beingexecuted by a computer. Generally, program modules include routines,programs, objects, components, data structures, etc. that performparticular tasks or implement particular abstract data types. Theinvention may also be practiced in distributed computing environmentswhere tasks are performed by remote processing devices that are linkedthrough a communications network. In a distributed computingenvironment, program modules may be located in both local and remotecomputer storage media including memory storage devices.

With reference to FIG. 1, an exemplary system for implementing theinvention includes a general purpose computing device in the form of acomputer 110. Components of computer 110 may include, but are notlimited to, a processing unit 120, a system memory 130, and a system bus121 that couples various system components including the system memoryto the processing unit 120. The system bus 121 may be any of severaltypes of bus structures including a memory bus or memory controller, aperipheral bus, and a local bus using any of a variety of busarchitectures. By way of example, and not limitation, such architecturesinclude Industry Standard Architecture (ISA) bus, Micro ChannelArchitecture (MCA) bus, Enhanced ISA (EISA) bus, Video ElectronicsStandards Association (VESA) local bus, and Peripheral ComponentInterconnect (PCI) bus also known as Mezzanine bus.

Computer 110 typically includes a variety of computer readable media.Computer readable media can be any available media that can be accessedby computer 110 and includes both volatile and nonvolatile media,removable and non-removable media. By way of example, and notlimitation, computer readable media may comprise computer storage mediaand communication media. Computer storage media includes both volatileand nonvolatile, removable and non-removable media implemented in anymethod or technology for storage of information such as computerreadable instructions, data structures, program modules or other data.Computer storage media includes, but is not limited to, RAM, ROM,EEPROM, flash memory or other memory technology, CD-ROM, digitalversatile disks (DVD) or other optical disk storage, magnetic cassettes,magnetic tape, magnetic disk storage or other magnetic storage devices,or any other medium which can be used to store the desired informationand which can be accessed by computer 110. Communication media typicallyembodies computer readable instructions, data structures, programmodules or other data in a modulated data signal such as a carrier waveor other transport mechanism and includes any information deliverymedia. The term “modulated data signal” means a signal that has one ormore of its characteristics set or changed in such a manner as to encodeinformation in the signal. By way of example, and not limitation,communication media includes wired media such as a wired network ordirect-wired connection, and wireless media such as acoustic, RF,infrared and other wireless media. Combinations of the any of the aboveshould also be included within the scope of computer readable media.

The system memory 130 includes computer storage media in the form ofvolatile and/or nonvolatile memory such as read only memory (ROM) 131and random access memory (RAM) 132. A basic input/output system 133(BIOS), containing the basic routines that help to transfer informationbetween elements within computer 110, such as during start-up, istypically stored in ROM 131. RAM 132 typically contains data and/orprogram modules that are immediately accessible to and/or presentlybeing operated on by processing unit 120. By way of example, and notlimitation, FIG. 1 illustrates operating system 134, applicationprograms 135, other program modules 136, and program data 137.

The computer 110 may also include other removable/non-removable,volatile/nonvolatile computer storage media. By way of example only,FIG. 1 illustrates a hard disk drive 141 that reads from or writes tonon-removable, nonvolatile magnetic media, a magnetic disk drive 151that reads from or writes to a removable, nonvolatile magnetic disk 152,and an optical disk drive 155 that reads from or writes to a removable,nonvolatile optical disk 156 such as a CD ROM or other optical media.Other removable/non-removable, volatile/nonvolatile computer storagemedia that can be used in the exemplary operating environment include,but are not limited to, magnetic tape cassettes, flash memory cards,digital versatile disks, digital video tape, solid state RAM, solidstate ROM, and the like. The hard disk drive 141 is typically connectedto the system bus 121 through an non-removable memory interface such asinterface 140, and magnetic disk drive 151 and optical disk drive 155are typically connected to the system bus 121 by a removable memoryinterface, such as interface 150.

The drives and their associated computer storage media discussed aboveand illustrated in FIG. 1, provide storage of computer readableinstructions, data structures, program modules and other data for thecomputer 110. In FIG. 1, for example, hard disk drive 141 is illustratedas storing operating system 144, application programs 145, other programmodules 146, and program data 147. Note that these components can eitherbe the same as or different from operating system 134, applicationprograms 135, other program modules 136, and program data 137. Operatingsystem 144, application programs 145, other program modules 146, andprogram data 147 are given different numbers here to illustrate that, ata minimum, they are different copies. A user may enter commands andinformation into the computer 110 through input devices such as akeyboard 162 and pointing device 161, commonly referred to as a mouse,trackball or touch pad. Other input devices (not shown) may include amicrophone, joystick, game pad, satellite dish, scanner, or the like.These and other input devices are often connected to the processing unit120 through a user input interface 160 that is coupled to the system bus121, but may be connected by other interface and bus structures, such asa parallel port, game port or a universal serial bus (USB). A monitor191 or other type of display device is also connected to the system bus121 via an interface, such as a video interface 190. In addition to themonitor, computers may also include other peripheral output devices suchas speakers 197 and printer 196, which may be connected through anoutput peripheral interface 195. Of particular significance to thepresent invention, a camera 163 (such as a digital/electronic still orvideo camera, or film/photographic scanner) capable of capturing asequence of images 164 can also be included as an input device to thepersonal computer 110. Further, while just one camera is depicted,multiple cameras could be included as an input device to the personalcomputer 110. The images 164 from the one or more cameras are input intothe computer 110 via an appropriate camera interface 165. This interface165 is connected to the system bus 121, thereby allowing the images tobe routed to and stored in the RAM 132, or one of the other data storagedevices associated with the computer 110. However, it is noted thatimage data can be input into the computer 110 from any of theaforementioned computer-readable media as well, without requiring theuse of the camera 163.

The computer 110 may operate in a networked environment using logicalconnections to one or more remote computers, such as a remote computer180. The remote computer 180 may be a personal computer, a server, arouter, a network PC, a peer device or other common network node, andtypically includes many or all of the elements described above relativeto the computer 110, although only a memory storage device 181 has beenillustrated in FIG. 1. The logical connections depicted in FIG. 1include a local area network (LAN) 171 and a wide area network (WAN)173, but may also include other networks. Such networking environmentsare commonplace in offices, enterprise-wide computer networks, intranetsand the Internet.

When used in a LAN networking environment, the computer 110 is connectedto the LAN 171 through a network interface or adapter 170. When used ina WAN networking environment, the computer 110 typically includes amodem 172 or other means for establishing communications over the WAN173, such as the Internet. The modem 172, which may be internal orexternal, may be connected to the system bus 121 via the user inputinterface 160, or other appropriate mechanism. In a networkedenvironment, program modules depicted relative to the computer 110, orportions thereof, may be stored in the remote memory storage device. Byway of example, and not limitation, FIG. 1 illustrates remoteapplication programs 185 as residing on memory device 181. It will beappreciated that the network connections shown are exemplary and othermeans of establishing a communications link between the computers may beused.

The exemplary operating environment having now been discussed, theremaining parts of this description section will be devoted to adescription of the program modules embodying the invention.

System Overview

The present invention provides a system and method that optimizesapplication software, such as PowerPoint®, for the capabilities ofspecific video hardware, thereby improving software performance andreliability. This system and method validates the different functions ofa 3D acceleration capable video card, decides whether to use graphicsacceleration hardware and optimizes the software application toselectively use the functions that work on the specific videoacceleration card. This procedure is typically run every time just asthe application software initializes. The invention can be couched inthe terms of use with a graphics presentation program, such asPowerPoint®, and a driver database such as Direct3D®. However, thosewith ordinary skill in the art will realize that the system and methodaccording to the present invention could be used with any graphicsintensive application program that employs the use of graphicsacceleration hardware in combination with any graphics driver library.

By way of example, FIG. 2 provides a general system schematic of how asoftware application 200, such as PowerPoint®, typically interfaces witha 3D graphics accelerator. In this case, the PowerPoint® softwareapplication includes a slideshow module 202 which presents the “slides”in the user's presentation as a cohesive package. This slideshow module202 interfaces with a drawing layer 204 that provides drawing commandsto display the aforementioned slides on a computer monitor or otherdisplay device. The drawing layer can either interface (typically via asoftware renderer 206) with the Central Processing Unit (CPU) 208 of theuser's computer to provide the drawing commands, or interface with agraphics accelerator renderer 210 which interfaces with the graphicsaccelerator 212, such as Direct3D®, to draw the respective drawings.

Referring now to FIG. 3, the graphics accelerator renderer 210, whichtypically resides in the application software, interfaces with thegraphics accelerator hardware 212 a and software 212 b through a drawer218, which converts drawing layer commands to graphics acceleratorcommands (in this case Direct3D® commands). Additionally, the graphicsaccelerator renderer 210 initializes the graphics accelerator 212hardware and software, via an initialization module 214 forcompatibility testing of graphics acceleration functions and determiningwhich functions are to be utilized with the application software. Thepresent invention automatically performs this compatibility testing withthe graphics acceleration hardware 212 a and software 212 b via a testmodule 216 as part of an initialization process. It should be noted thatit is typically desirable to use the graphics accelerator hardware 212 awith this invention rather than the software 212 b, since this hardwareprocesses 3D data much more efficiently than either the softwareresident in the graphics accelerator or the software renderer thatinterfaces with the system's CPU. The test module 216 also interfaceswith a drawer and texture manager for the purpose of performing theaforementioned compatibility testing of the graphics accelerationhardware of the video card.

In the most general sense, as shown in FIGS. 4A and 4B, the inventionperforms compatibility testing of the graphics hardware by firstchecking to see if graphics acceleration is selected or enabled (processaction 402), checking to see if the video card is on a list of cardswith known problems (process action 404), and verifying that thegraphics accelerator driver library initializes the graphicsacceleration hardware of the video card successfully (process action406). The invention also checks to see if sufficient video memory isavailable to perform the various remaining compatibility tests (processaction 408) and verifies that calls to the video card hardware aresuccessful (process action 410). In each of these process actions if theparticular process action indicates that there is a problem with thecompatibility of the video card and the application software thedecision is made not to use the graphics acceleration hardware of thevideo card (process action 426). Additionally, if the above processactions indicate no compatibility problems, the options to be used withthe video card are selected and stored (process action 412), as shown inFIG. 4B. Then sub-pixel positioning, which is described in more detailbelow, is tested (process action 414). If this test is successful, thenpixel offset values are stored (process action 416). If the test is notsuccessful, the decision is made not to use graphics acceleration(process action 426). A test is then conducted to determine whether theopacity function performs correctly when used with the graphicsaccelerator (process action 418). Again, if the opacity function doesnot work correctly then the procedure is exited with a determination notto use the graphics acceleration hardware of the video card (processaction 426). If the opacity test is passed, however, a test is conductedfor color replacement (process action 420). If the color replacementtest does not pass, another test wherein a fog function is used toperform color replacement is conducted (process action 422), and if thistest also does not pass then the decision is made not to use graphicsacceleration (process action 426). If the initial color replacement test(process action 420) passes, then the decision is again made to usegraphics acceleration (process action 424), and the system and methodaccording to the present invention continues to run the applicationsoftware. Those with ordinary skill in the art will realize that it isnot necessary to perform all of the above process actions in a specificorder. Additionally, other types of tests could be performed to ensuresoftware application and video card hardware compatibility before thesoftware application is used to render 3D graphics.

For a more detailed discussion of the various tests performed todetermine the compatibility between the application software and thegraphics acceleration hardware of the video card, FIG. 5A is referredto. To begin with, the software program, such as PowerPoint®, typicallyinitially examines whether the user has turned off the hardware graphicsacceleration option or the software has previously disabled the optionbased on a previous failed attempt to use graphics acceleration, asshown in process action 502. To this end, a graphical user interface, asshown in FIG. 6, may be made available to the user to disable thegraphics acceleration option by checking a box 602 with a computer inputdevice, thereby setting a flag in the software. The application softwaremay also set this flag if a previous attempt to use graphicsacceleration fails, as will be discussed in further detail later. Thesoftware then examines whether this flag is set in the software. If so,the system and method according to the present invention determines thatthe graphics acceleration hardware is not to be used, and exits theprocedure (process action 550). If the user has not turned off thehardware graphics acceleration option of the software, the systemcontinues to examine the feasibility of using the graphics accelerationhardware of the video card as described below.

In addition, in one embodiment, the invention then examines whether thevideo card resident in the user's computer hardware is on a list ofcards that is known to be incompatible when used with the softwareprogram, as shown in process action 504. This test is performed by theapplication software by maintaining a database of video cards and/orvideo card drivers that are known to be incompatible with theapplication software and checking to see if the video card and/or videocard drivers resident in the user's computer is in the database. If thecard or driver is determined to be incompatible with the software bythis procedure, the present invention does not use the graphicsacceleration hardware of the video card, and exits the testing steps(process action 550).

The system and method according to the present invention then attemptsto initialize the graphics acceleration functionality by initializing adrawer in the application software, ensuring it loads successfully andmaking calls to it (process action 506). If the initialization isunsuccessful, the system determines that the graphics accelerationhardware of the video card is not to be used and exits the procedure(process action 550). If the initialization is successful, the systemand method according to the present invention proceeds to check thevideo memory of on-screen and off-screen buffers of the video card byrequesting the video memory for them (process action 508). If this callis unsuccessful the decision is made not to use the graphicsacceleration hardware (process action 550). If the call is successful,the system continues its checks.

As shown in process action 510, the system then requests to access thevideo card hardware through the hardware abstraction layer (HAL). TheHAL is a software interface to various 3D features of the video card. Ifthe hardware device cannot be accessed, the decision is made not to usethe graphics acceleration hardware (process action 550). If access tothe graphics acceleration hardware is successful, the system and methodaccording to the present invention proceeds to set the graphicsacceleration options necessary to support the application software(process action 512). These options are saved in memory and accessed asthe application software runs. Such options include, for example, howtextures are to be drawn and the selection of various filtering options.

As shown in FIG. 5B, the system then checks if there is sufficient videomemory to perform the remainder of the compatibility tests between theapplication software and the graphics hardware (process action 514).During these tests, test patterns and computations are written tooff-screen video memory. Hence, both the on-screen and off-screenbuffers are used. Using dual buffers is a known graphics technique fordelaying the effect of graphics operations, typically implemented byrendering graphics operations into an off-screen buffer. The contents ofthe buffer are then copied to the screen in a single operation. To drawthe test patterns and to perform the applicable calculations of thepresent invention, there typically must be enough video memory for theon-screen and off-screen buffers plus the number of textures used in thecompatibility tests (in a tested embodiment this was eight textures ofsize 128×128 pixels).

The system and method according to the present invention then creates atexture (process action 516) and fills it with a test pattern (processaction 518). The test pattern preferably consists of a black backgroundand diagonal line of pixels of all white color except the first threepixels which are red, green and blue. This test pattern is typicallydrawn to the off-screen buffer at position 1,1 (process action 522).

Next the sub-pixel positioning is tested. Like most computer graphicimaging applications, PowerPoint®, which was used in a tested embodimentof the present invention, offers “sub-pixel positioning” to achievesmooth movement in any transformation of displayed images or graphics.Sub-pixel positioning refers to the ability to “move” an image less thanone pixel's distance. Sub-pixel positioning is essential because of theeye's ability to detect even the slightest jump or inconsistency inmotion. Due to the “finite” amount of pixels in an image (e.g. 720horizontal pixels in video), fine detail often becomes “soft” (e.g. notas fine) when the image is moved or rotated. Sub-pixel positioning iscritical to presentation software, such as PowerPoint®, because theclarity of text on the computer screen is critical to the readability ofthe presentation.

In general, this sub-pixel positioning test consists of 1) drawing thetest image to off-screen video memory (process action 520); 2)determining where the pixels for the test image were drawn using theexpected pixel boundary and color (process action 522); 3) calculatingthe offset values between where the pixels were expected to be drawn andwhere they were actually drawn (process action 522); and 4) determiningwhether the offset values are acceptable to correct for this offsetduring 3D graphics rendering (process action 524). If the offset valuesare not acceptable, different offset values may be tried until theoffset values are below the predetermined threshold (as shown in processactions 526 and 527).

Specifically, the test pattern is preferably drawn to off-screen videomemory and it is assumed that the center of a test pixel is at 0.5, 0.5(further assuming a pixel matrix of 128×128 pixels and having an originof 0,0). The pixel values of the test pixel are then read back from theoff-screen video memory. If the values match what is expected (i.e, thevalues are below a given threshold) then this sub-pixel positioning isused (stored in memory to correct for pixel position during theapplication run time). That is, if the difference in pixel colors fromwhat is expected and what is read from the screen buffer is less thanthe threshold, then this sub-pixel positioning is used. One exemplaryformula that may be used for this purpose is(abs(r_actual−r_expected)+abs(g_actual−g_expected)+abs(b_actual−b_expected)),where where r_actual represents the amount of red in the actual pixelcolor, r_expected represents the amount of red in the expected pixelcolor, where g_actual represents the amount of green in the actual pixelcolor, g_expected represents the amount of green in the expected pixelcolor, and where b_actual represents the amount of blue in the actualpixel color, b_expected represents the amount of blue in the expectedpixel color. For example, using a threshold of 24, if a RGB color of(255,0,0) is expected and an actual color of (250,1,0) is read, then thedifference in colors between the actual and expected values is 6 so thissub-pixel positioning is accepted. If the read values do not match(i.e., the values are over the threshold) then other pixel offset valuesare tried in an attempt to determine the actual pixel offset until amatch is found. This process may be repeated for a number of cycles. Ina tested embodiment four cycles were completed using pixel offset valuesof 0.5+{fraction (1/16)}, 0, 0+{fraction (1/16)}, and 0.5 from theorigin of each pixel as drawn to the offscreen video memory by thegraphics hardware.

If the difference in value between the expected pixel color and the readpixel colors is greater than the threshold (e.g., 24, in the aboveexample) for all (e.g., four) cycles using different offset values, thenthe minimization filter and maximization filters are turned off (processaction 528) and additional cycles (e.g., four) of the above process arerepeated again, as shown in process actions 530 to 537. If this stilldoes not bring the difference between the expected pixel color and theread pixel values under the threshold then the determination is made notto use the graphics acceleration as the resultant sub-pixel positioningwill result in noticeable distortion or blurring of the 3D image(process action 550).

As shown in FIG. 5C, if the sub-pixel positioning is determined to besuitable, the system and method according to the present invention thentests to see if the graphics acceleration hardware can perform theopacity function. This is typically done by drawing a shape whilesetting the opacity of a pixel to a prescribed value such as 50%(process action 538). The color value of the pixel is then read from thescreen buffer (process action 540). If the color has not changed then itis assumed that the card cannot perform the opacity function andgraphics acceleration is not used (process action 550).

The color replacement function is then tested. Several conventional 3Dacceleration techniques can be used for this purpose. One of thesetechniques tests color replacement by replacing all of the color in atexture by another color without changing the opacity values. Forexample, a texture is drawn in red by telling the card to use DiffuseColor for the texture and to ignore the original color of the texture(process action 542). The color of the pixels is then read (processaction 546). If the color is not as expected then an attempt is made todetermine whether a fog function can be used to mimic the colorreplacement function (process action 548). If this fails, then graphicsacceleration function is not used (process action 550). However, if allof the above steps were successful then the hardware graphicsacceleration function is used (process action 552).

Even with the above-specified compatibility tests, there is a chancethat the hardware will fail. To mitigate this possibility, the presentinvention automatically disables graphics acceleration if the graphicsaccelerator hardware crashes or hangs during the running of the softwareapplication. The next time the user launches application software,slideshow in the case of PowerPoint® application software, it does notcrash. To do this, a flag is set when slideshow is started and unset itonce the slideshow successfully ends. However, if slideshow does notsuccessfully end then the flag is not unset. The next time slideshow islaunched, the program notes that the flag is set and graphicsacceleration is not used.

While the invention has been described in detail by specific referenceto preferred embodiments thereof, it is understood that variations andmodifications thereof may be made without departing from the true spiritand scope of the invention. For example, the system and method describedabove is not limited to the specific tests discussed herein. Othergraphics acceleration functions could be tested like multi-texturing,colored lighting, texture size restrictions, and any other hardwarefeatures that are used

Wherefore, having thus described the present invention, what is claimedis:
 1. A computer-implemented process for optimizing the performance ofa graphics intensive software application while using graphicsacceleration hardware of a video card comprising the process actions of:attempting to initialize the graphics acceleration hardware; testing thecompatibility of one or more software functions requiring the graphicsacceleration hardware if the attempt to initialize is successful whereinthe testing process action comprises at least one of: verifying thegraphics acceleration hardware of the video card is not on a list ofvideo cards with known incompatibilities with the application software;verifying that the application software initializes the interface to thegraphics acceleration hardware of the video card successfully; verifyingthat there is sufficient memory on the video card to conduct the tests;verifying that the calls to the video card hardware are successful;verifying that a sub-pixel positioning function to correct for offsetsbetween actual pixel positions of a texture drawn to video card memoryand expected pixel position of a texture drawn to video card memory isoperational and wherein the process action of verifying that sub-pixelpositioning function is operational comprises: drawing a test image tovideo memory wherein the process action of drawing a test image to videomemory comprises drawing a test image that consists of a blackbackground and a diagonal line of pixels of all white color except forthe first three pixels which are red, green and blue; determining wherethe pixels for the test image were drawn using the expected pixelboundary and color of each pixel considered; calculating the offsetvalues between where the pixels were expected to be drawn and where theywere actually drawn; and determining whether the calculated offsetvalues are acceptable to correct for the pixel offset during 3D graphicsrendering; verifying that an opacity function, which emulates opacity ofrendered objects in the application software, is operational; andverifying that a color replacement function, that replaces colors of arendered object in the application software, is operational; andestablishing the optimum utilization of the graphics accelerationhardware based upon the results of the compatibility testing.
 2. Theprocess of claim 1, further comprising the process actions of: initiallydetermining whether a user has turned off a hardware accelerationoption; and not using the graphics acceleration hardware and foregoingthe attempting, testing and establishing process actions if the hardwareacceleration option is turned off.
 3. The process of claim 1, furthercomprising the process actions of: initially determining whether theapplication software has turned off a hardware acceleration option; andnot using the graphics acceleration hardware and foregoing theattempting, testing and establishing process actions if the hardwareacceleration option is turned off.
 4. The process of claim 1, furthercomprising the process actions of: setting a flag in the applicationsoftware each time the application software is run using the graphicsacceleration hardware; clearing the flag when the software applicationsuccessfully completes using the graphics acceleration hardware; andchecking if the flag is set each time the software applicationinitializes, using graphics acceleration if the flag is not set, and,not using the graphics acceleration hardware and foregoing theattempting, testing and establishing process actions if the flag is set.5. The process of claim 1, further comprising the process action of notusing graphics acceleration and foregoing further tests if any of theverification process actions fail.
 6. The process of claim 1 wherein theprocess action of verifying the graphics acceleration video card is noton a list of video cards with known incompatibilities further comprises:creating a database of all video cards known to be incompatible with thesoftware program, each video card having a unique identifier; extractingan identifier of the graphics video card; comparing the identifier onthe video card to the identifiers in the data base; and determining thatthe video card is incompatible with the application software if a uniqueidentifier on the card matches an identifier in the database.
 7. Theprocess of claim 6 wherein the unique identifier is a video cardidentifier.
 8. The process of claim 6 wherein the unique identifier is avideo card driver identifier.
 9. The process of claim 1 wherein theprocess action of verifying that the application software initializesthe interface to the graphics accelerator successfully comprises:initializing a drawer, which converts software application drawingcommands to graphics accelerator commands, in the application software;ensuring the drawer initializes successfully; and making calls to thedrawer to ensure the drawer.
 10. The process of claim 1 wherein theprocess action of verifying that the calls to the video card hardwareare successful, comprises: requesting access from the softwareapplication to the graphics acceleration hardware of the video cardthrough a hardware abstraction layer (HAL), which is a softwareinterface to various 3D features of the video card; accessing thegraphics acceleration hardware of the video card to set graphicsacceleration options necessary to support the application software; andsaving the options to memory to be accessed as the application softwareruns.
 11. The process of claim 10 wherein the options comprise at leastone of: how textures are to be drawn; and how textures are to befiltered.
 12. The process of claim 1 wherein the process action ofverifying that there is sufficient memory on the video card to conductthe tests comprises: determining the amount of video memory is requiredto draw textures used in testing; determining the amount of video memoryis required to perform desired test calculations; adding the amount ofvideo memory required to draw textures used in testing to the amount ofvideo memory required to perform the desired test calculation;identifying the amount of video memory available from the video card;comparing the summed video memory to the amount of video memoryavailable; and declaring the amount of video memory to be sufficient ifthe amount of video memory available exceeds the summed video memory.13. The process of claim 1 wherein the test pattern is drawn to anoff-screen buffer at pixel position 1,1 in array of pixels with originof 0,0.
 14. The process of claim 1 wherein the process action ofcalculating the offset values between where the pixels were expected tobe drawn and where they were actually drawn comprises using the formulaabs(r_actual−r expected)+abs(g_actual−g_expected)+abs(b_actual−b_expected), where r_actualrepresents the amount of red in the actual pixel color, r_expectedrepresents the amount of red in the expected pixel color, where g_actualrepresents the amount of green in the actual pixel color, g_expectedrepresents the amount of green in the expected pixel color, and whereb_actual represents the amount of blue in the actual pixel color,b_expected represents the amount of blue in the expected pixel color.15. The process of claim 1 wherein the process action of determiningwhether the offset values are acceptable to correct for the pixel offsetduring 3D graphics rendering comprises comparing the offset values to apredetermined threshold.
 16. The process of claim 1 further comprisingrepeating, for a given number of cycles, drawing a test image to videomemory; determining where the pixels for the test image were drawn usingthe expected pixel boundary and color plus a new offset value;calculating the offset values between where the pixels were expected tobe drawn and where they were actually drawn; and determining whether thecalculated offset values are acceptable to correct for the pixel offsetduring 3D graphics rendering.
 17. The process of claim 16 wherein it isdetermined that the graphics acceleration hardware shall not be used ifoffset values are not acceptable to correct for pixel offset in thegiven number of cycles.
 18. The process of claim 16 wherein the processaction of determining whether the offset values are acceptable tocorrect for the pixel offset during 3D graphics rendering comprisescomparing the offset values to a predetermined threshold that isdetermined by the difference between the sum of the absolute value ofthe actual pixel color value minus the expected pixel color value forred, green and blue.
 19. The process of claim 18 wherein thepredetermined threshold is
 24. 20. The process of claim 18 wherein thenew offset value is one of 0, 0.5+{fraction (1/16)}, 0.5 and 0+{fraction(1/16)} from the origin of each pixel as drawn to the offscreen videomemory by the graphics hardware.
 21. The process of claim 16 furthercomprising the process actions of: turning off minimization andmaximization filters if the calculated offset values are not acceptablewithin the given number of cycles; and for another given number ofcycles: drawing a test image to video memory; determining where thepixels for the test image were drawn using the expected pixel boundaryand color plus a new offset value; calculating the offset values betweenwhere the pixels were expected to be drawn and where they were actuallydrawn; and determining whether the calculated offset values areacceptable to correct for the pixel offset during 3D graphics rendering.22. The process of claim 21 wherein it is determined that the graphicsacceleration hardware shall not be used if the calculated offset valuesare never acceptable to correct for pixel offset within the number ofcycles.
 23. The process of claim 1 wherein the process action ofverifying that sub-pixel positioning is functional further comprises:turning off minimization and maximization filters; drawing a test imageto video memory; determining where the pixels for the test image weredrawn using the expected pixel boundary and color; calculating theoffset values between where the pixels were expected to be drawn andwhere they were actually drawn; and determining whether the calculatedoffset values are acceptable to correct for the pixel offset during 3Dgraphics rendering.
 24. A computer-implemented process for optimizingthe performance of a graphics intensive software application while usinggraphics acceleration hardware of a video card comprising the processactions of: attempting to initialize the graphics acceleration hardware;testing the compatibility of one or more software functions requiringthe graphics acceleration hardware if the attempt to initialize issuccessful, wherein the testing process action comprises at least oneof: verifying the graphics acceleration hardware of the video card isnot on a list of video cards with known incompatibilities with theapplication software; verifying that the application softwareinitializes the interface to the graphics acceleration hardware of thevideo card successfully; verifying that there is sufficient memory onthe video card to conduct the tests; verifying that the calls to thevideo card hardware are successful; verifying that a sub-pixelpositioning function to correct for offsets between actual pixelpositions of a texture drawn to video card memory and expected pixelposition of a texture drawn to video card memory is operational;verifying that an opacity function, which emulates opacity of renderedobjects in the application software, is operational; and verifying thata color replacement function, that replaces colors of a rendered objectin the application software, is operational wherein the process actionof verifying that color replacement function is operational furthercomprises the process actions of: drawing a texture of a given color;replacing all of the color in the texture by a second color; reading thecolor of the pixels in the texture after replacing all of the color inthe texture by the second color; and comparing the color of the pixelsin the texture to the second color; and establishing the optimumutilization of the graphics acceleration hardware based upon the resultsof the compatibility testing.
 25. The process of claim 24 furthercomprising the process action of declaring that color replacementfunction as non-operational if the color of the pixels in the textureare not the same as the second color.
 26. The process of claim 24further comprising the process action of attempting to mimic colorreplacement using the fog function.
 27. The process of claim 24 whereinthe process action of verifying that opacity function is functionalcomprises the process actions of: drawing a texture while setting theopacity to a predetermined value; reading pixel color of the texturefrom the screen buffer; determining that the opacity is not functionalif the read color is not as expected.